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Art Unit: 2627 

(1) Real Party in Interest 

A statement identifying the real party in interest is contained in the brief. 

(2) Related Appeals and Interferences 

The examiner is not aware of any related appeals, interferences, or judicial proceedings 
which will directly affect or be directly affected by or have a bearing on the Board's decision in 
the pending appeal. 

(3) Status of Claims 

The statement of the status of claims contained in the brief is correct. 

(4) Status of Amendments After Final 
No amendment after final has been filed. 

(5) Summary of Claimed Subject Matter 

The summary of claimed subject matter contained in the brief is correct. 

(6) Grounds of Rejection to be Reviewed on Appeal 

The appellant's statement of the grounds of rejection to be reviewed on appeal is correct. 

(7) Claims Appendix 

The copy of the appealed claims contained in the Appendix to the brief is correct. 

(8) Evidence Relied Upon 



4,985,856 



KAUFMAN ET AL 



1-1991 



6,721,449 



KRISHNAMACHARI 



4-2004 



6,691,126 



SYEDA-MAHMOOD 



2-2004 



6,711,288 



KIM ETAL 



3-2004 
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Moghaddam et al. "Image Retrieval with Local and Spatial Queries" Proceedings of the 2000 

Intemational Conference on Image Processing, vol2 (Sep 2000), pp. 542-545. 

Swain et al. "Color Indexing" Intemational Journal of Computer Vision, vol7, no. 1 (1991), pp. 

11-32. 

(9) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims (reprinted 
from the Final Rejection dated II 1 3/05): 

Claim Rejections - 35 USC §103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claims 1-3,5-11, and 19 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
U.S. Patent 6,721,449 by Krishnamachari in view of "Image Retrieval with Local and Spatial 
Queries" by Moghaddam et al. ("Moghaddam"). 

Regarding claims 1 and 19, Krishnamachari discloses a method of measuring color 
consistency comprising: 

obtaining a first 2-D image and a second 2-D image of an object (figure 1 : a target image 
and a reference image of an object from a database are obtained); 
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subdividing the first image into a first set of partitions and the second image into a second 
set of image partitions, each partition having a color (i.e. each of the first and second images are 
partitioned into pixels ("partitions"), each pixel having a color); 

selecting a first subset of image partitions in the first set of image partitions (e.g. a first 
image block as one of 4x4, 8x8, or 16x16 image blocks is a first subset of pixels in the target 
image) and a second subset of image partitions in the second set of image partitions (e.g. a 
second image block as one of 4x4, 8x8, or 16x16 image blocks is a second subset of pixels in the 
reference image) (see column 4, line 59 through column 5, line 13); 

assigning each image partition in the first subset and each image partition in the second 
subset a color value corresponding to the color of the image partition (column 4, lines 33-58: 
pixels in each block are assigned CIELAB or CIELUV color values based on their respective 
colors); 

placing each image partition in the first subset in one of a first series of histogram 
subdivisions and each image partition in the second subset in one of a second series of histogram 
subdivisions based on the color value of each image partition (column 5, lines 14-34: for each of 
the first and second image blocks, an histogram is generated whereby pixels in each block are 
accumulated into histogram subdivisions based on color values of the pixels); 

comparing the first series of histogram subdivisions to the second series of histogram 
subdivisions (column 5, lines 25-34: the histograms of corresponding image blocks in the target 
and reference images are compared based on the "proportion of occurrences" of the colors 
therein, thereby effecting a comparison of the histogram bins ("subdivisions") for corresponding 
blocks in each of the two images); 
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processing the second image based on whether the first series of histogram subdivisions 
and the second series of histogram subdivisions have a similarity (column 6, lines 19-29: if the 
first and second series of histograms subdivisions are similar (as well as other corresponding 
series pairs), then the reference image is identified as being similar to the target image, and is 
processed so that it is displayed in a list of similar images), 

Krishnamachari is silent to selecting the blocks ("subsets") of pixels ("image partitions") 
"based upon criteria related to a 3-D region of the object" and processing "the region" based on 
whether the pair of series of histogram subdivisions is similar. 

Moghaddam discloses a content-based image retrieval system, similar to that of 
Krishnamachari, in which database images that are substantially similar to a target image are 
retrieved. Moghaddam, like Krishnamachari, utilizes local color histograms in order to effect a 
comparison between a target and a reference image. 

Moghaddam also discloses the feature of allowing a user to specify regions of interest 
which serve as search criteria and lead to "a more powerful search engine " See § 1 . As shown in 
figure 1, a user can select regibn(s) of interest corresponding to three-dimensional objects in a 
scene. Those regions of interest are then processed using local histogram measures, as shown in 
figure 2. The histograms of those regions are then matched to histograms of the reference images 
in order to find a sufficiently matching image. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Krishnamachari by Moghaddam to select the subsets of image partitions "based upon 
criteria related to a 3-D region of the object" and process "the region" based on whether the pair 
of series of histogram subdivisions is similar since Moghaddam discloses that selecting subsets 
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of image partitions in a target image based on a region of a 3-D object according to a user's 
preference and then processing (e.g. retrieving and displaying) the region based on the similarity 
between histogram subdivisions of the target and reference images produces "a more powerful 
search engine"; a user is allowed to specify arbitrary regions of objects to be retrieved rather than 
relying on a computer to specify the regions (see Abstract and § 1). 

Regarding claim 2, Krishnamachari discloses that the obtaining step comprises obtaining 
images of a scene as the object (figure 1 : "target image" and reference image(s) in the "image 
database" are images of scenes). 

Regarding claim 3, Krishnamachari discloses subdividing the first and second image into 
pixels (i.e. the images are digital images, which are divided into pixels). 

Regarding claim 5, Moghaddam discloses that only the pixels located in the region are 
selected (see figure 2(b)). 

Regarding claim 6, Krishnamachari discloses assigning an array value as the color value 
(column 4, lines 33-59: e.g. LUV array value). 

Regarding claim 1, Krishnamachari discloses assigning a 3-D array value (i.e. LUV is a 
3-D array). 

Regarding claim 8, Krishnamachari discloses that utilizing R, G, and B values is 
conventional (column 1, lines 35-45). 

Regarding claim 9, Krishnamachari discloses that the RGB values are typically 24 bits, 
so that the values of each color range from 0-255 (column 1, lines 35-45). 
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Regarding claim 10, Krishnamachari discloses combining the R, G, and B values into a 
smaller number of agglomerate values (column 1, lines 55-64). 

Regarding claim 11, Krishnamachari discloses that combining portions of the first and 
second histogram subdivisions into a first and second series of histogram partitions is 
conventional (column 1, lines 47-67: the 24-bit RGB values, representing millions of colors, are 
quantized into 64 colors so that the millions of histogram subdivisions for each image are 
combined to form 64 partitions ^ in other words, portions of the first and second histogram 
subdivisions (i.e. different groups of the histogram bins) are combined into a series of histogram 
partitions (i.e. the groups each combine to from one of 64 partitions)). 

Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over Krishnamachari in 
view of Moghaddam, and further in view of U.S. Patent 4,985,856 by Kaufinan et al. 
("Kaufman"). 

Regarding claim 4, Krishnamachari and Moghaddam are silent to selecting the subset 
based on criteria related to a voxel region of the object. 

Kaufman discloses a 3-D image storage and retrieval system. Rather than storing 
conventional 2-D image data, Kaufman's system is operative to store and retrieve 3-D image 
data composed of voxels. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Krishnamachari and Moghaddam by Kaufinan to select a the subsets based on criteria 
related to a voxel region, rather than criteria related to a pixel region, since Kaufman discloses 
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that it is desirable to represent images in 3-D with voxels and store and retrieve such images (see 
column 2, lines 56-66 and column 9, lines 45-52). 

Claim 15 is rejected under 35 U.S.C. 103(a) as being unpatentable over Krishnamachari 
in view of Moghaddam, and further in view of U.S. Patent 6,691,126 by Syeda-Mahmood. 

Regarding claim 15, Krishnamachari and Moghaddam are silent to the object including a 
Lamertian surface. 

Syeda-Mahmood discloses an image processing system for locating objects in a video 
database. In particular, Sydea-Mahmood discloses methods by which Lambertian surfaces are 
modeled and retrieved in a video database (see column 6, lines 41-48). 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Krishnamachari and Moghaddam by Syeda-Mahmood to choose the object to include 
a Lambertian surface, since Krishnamachari and Moghaddam disclose that any arbitrary object in 
an image may be obtained for retrieval purposes, and Syeda-Mahmood shows that it is 
conventional to retrieve objects that represent Lamertian surfaces. 

Claim 14 is rejected under 35 U.S.C. 103(a) as being unpatentable over Krishnamachari 
in view of Moghaddam, and further in view of "Color Indexing" by Swain et al. ("Swain"). 

Regarding claim 14, Krishnamachari is silent to comparing corresponding histogram 
subdivisions, as claimed. 
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Swain discloses the conventional manner in which histograms are compared. At § 3.1.1, 
Swain discloses that histograms are compared by corrq^aring corresponding histogram 
subdivisions via a "min" function to see if each subdivision contains at least one pixel. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Krishnamachari and Moghaddam by Swain to compare histograms as claimed, since 
Swain shows that comparing corresponding histogram subdivisions to ascertain whether pixels 
are present in each of the bins is a conventional technique for effecting a comparison of 
histograms. 

Claim 17 is rejected under 35 U.S.C. 103(a) as being unpatentable over Krishnamachari 
in view of Moghaddam, and further in view of U.S. Patent 6,71 1,288 by Kim et al. ("Kim"). 

Regarding claim 17, neither Krishnamachari nor Moghaddam discloses assigning a 
uniform region color to the entire region in the object. 

Kim discloses a method for designating a local representative color value for image 
regions in order to facilitate image retrieval. In particular, Kim discloses assigning uniform 
region colors to each of the image regions based on the local color histogram (see figure 2A, 
block S52). Then, the local representative color of each region is used to select an optimal 
searching algorithm. 

It would have been obvious to one of ordinary skill in the art at the time of the invention 
to modify Krishnamachari and Moghaddam by Kim to assign the region a uniform color since 
Kim discloses that utilizing local representative colors (i.e. uniform region colors) allows an 
optimal search algorithm to be automatically ascertained (see column 3, hnes 18-25). 



Application/Control Number: 09/987,918 
Art Unit: 2627 



Page 10 



(10) Response to Argument 

Regarding claim 1, Appellant presents the following arguments: 

(1) Krishnamachari does not obtain first and second 2-D images "of an object," as 
claimed (see pp. 5-6 of the Brief); 

(2) Krishnamachari does not "process the region," as claimed (see pp. 6-7 of the Brief); 

(3) one skilled in the art would not have been motivated to modify Krishnamachari to 
achieve the claimed invention (see pp. 7-8 of the Brief); 

(4) the combination of Krishnamachari fails to teach or suggest all of the claim 
limitations (see pp. 8-9 of the Brief); and 

(5) Moghaddam does not teach selecting the first and second subsets of image partitions 
"based upon criteria related to a 3-D region of the object," as claimed (see p. 9 of the Brief). 

Regarding the first argument, Applicant argues that Krishnamachari does not disclose 
obtaining first and second 2-D images of an object. AppUcant asserts that Krishnamachari' s 
image retrieval system "does not teach, describe or suggest that target image 101 and any of the 
images 111 (in the retrieval database) are of the same object" (see Brief, p. 5). 

It is true that Krishnamachari does not expressly disclose that any of the database images 
are of the same object as the target image. However, the converse is also true in that 
Krishnamachari does not expressly disclose that all the images in the database are not of the 
same object as the target image. Nevertheless, the stated purpose of Krishnamachari' s system is 
to retrieve images from a database that are most similar to a target image. 
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For example, Krishnamachari's system is operative to store a database of images of 
paintings; the user then provides a target image of a painting. The target image may be an 
unknown painting, and the user wishes identify it; or the user may be familiar with the painting 
and wants to know whether an image of the painting is in the database. Krishnamachari's system 
then compares a color histogram of the target image to a color histogram of each of the images in 
the database. The system then retrieves the images in the database that are most similar'fotKe 
target image based on the comparison. 

As mentioned above, Krishnamachari does not expressly state that the target image is of 
the same object as any of the database images. However, such a limitation is so easily inferred as 
to be implicit in Krishnamachari's disclosure. In the above example, the target image and a 
database image may very well be of the same object (i.e. painting). If that is the case, then the 
system will report that the target image and the database image of the same painting are 
substantially similar. On the other hand, if the painting in the target image is not found in any of 
the database images, then the system may not find any of the images in the database to be 
similar; however, there may be other images of paintings in the database that use similar colors 
or that are by the same artist so that the system does find a database image(s) to be similar, 
although not identical, to the target image. 

Such is the operation of Krishnamachari's system - it does not require that any of the 
database images are of the same object as the target image, but this limitation would have been 
implicit to those skilled in the art, since the purpose of Krishnamachari's retrieval system is to 
identify any database images that match a provided target image. Whether there exist any images 
in the database that are of the same object as contained in the target image will be determined by 
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the specific implementation and environment in which Krishnamachari's invention is practiced. 
Those skilled in the art, upon reading Krishnamachari's disclosure, would have easily known 
that, since the purpose of the system is to retrieve images similar to a target image, retrieving 
images that are identical and of the same object as contained in a target image was so well within 
scope of Krishnamachari's teachings as to be implicit therein. 

Regarding the second argument, Apphcant argues that Krishnamachari does not disclose 
processing a region, since Krishnamachari's disclosure is directed to processing entire images. 
As stated in the Final Rejection reprinted above, Krishnamachari is not considered to teach this 
feature. Therefore, the Moghaddam reference was relied upon as providing the requisite teaching 
and motivation for processing regions of interest (ROIs) in an image rather than entire images. 

Regarding the third argument. Applicant argues that one skilled in the art would not have 
been motivated to combine the teachings of Krishnamachari and Moghaddam to achieve the 
claimed invention. Examiner maintains that there is sufficient motivation to combine the 
teachings of the two references. Both are related to content-based image retrieval, and more 
specifically, both are related to content-based image retrieval involving the comparison of color 
histograms between images in a database and a target image. 

Applicant asserts that, in contrast to Krishnamachari and Moghaddam, "the present 
invention is used for determining color consistency of regions of images of the same object for 
use in 3-D modeUng" (Brief, p. 8). Examiner appreciates the differences between the disclosed 
invention and the teachings of Krishnamachari and Moghaddam, However, in the Examiner's 



Application/Control Number: 09/987,918 Page 13 

Art Unit: 2627 

opinion, the claimed invention is an obvious combination of the prior art image query and 
retrieval systems taught by Krishnamachari and Moghaddam. That is, the invention, as claimed 
in claim 1, does not differentiate a method of measuring color consistency for image retrieval 
from a method of measuring color consistency for 3-D modeling. The claim merely calls for 
comparing the histograms of an object region in two images of the same object, and then 
processing the region based whether the histograms are similar. In the claim, there are no details 
regarding how the image is "processed" that would differentiate the claim as a method involving 
3-D modeling from a method involving image retrieval. 

Regarding the fourth argument, Appellant asserts that the combination of 
Krishnamachari and Moghaddam does not teach all of the claimed limitations because 
Moghaddam does not teach or suggest obtaining first and second 2-D images "of an object," as 
claimed. In response, this limitation was addressed for the first argument above, and 
Krishnamachari was considered to disclose this feature at least implicitly for the reasons given. 

Appellant also asserts that Moghaddam does not teach or suggest, "processing" the 
region based on whether the first and second series of histogram subdivisions have a similarity. 
In response, Moghaddam is considered to "process" the region, as claimed. As stated in the Final 
Rejection reprinted above, Moghaddam "processes" the region of interest at least by retrieving it 
and displaying it (e.g. figs. 5 and 6). In other words, upon ascertaining a match for the region of 
interest, a region corresponding to the match is retrieved from the database and then displayed 
for a user to view. Retrieving and displaying a matching region is considered to be "processing" 
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the region since the claim does not specify any details of the "processing" and only nominally 
recites "processing the region." 

Regarding the fifth argument. Appellant asserts that Moghaddam does not teach selecting 
the first and second subsets of image partitions "based upon criteria related to a 3-D region of the 
object," as claimed (see p. 9 of the Brief). Appellant asserts that Moghaddam instead utihzes 2-D 
"spatial information" in selecting a subset of the partitions, as shown in figure 4. 

Examiner agrees that Moghaddam does utilize 2-D spatial information in selecting, 
however, when the claim limitation is given its broadest reasonable scope, it does read on 
Moghaddam' s process for selecting the subsets of image partitions. 

The claim calls for "selecting a first . . . and a second subset . . . based upon a criteria 
related to a three-dimensional region of the object." {emphasis added) In other words, the 
subsets are selected using criteria that are somehow related to a 3-D region of the object. Figure 
1 of Moghaddam illustrates such a selection. It shows a dog sitting underneath an archway. The 
archway and the dog are both real-world 3-D objects. The image of the dog and the archway is 
represented by a 2-D configuration of pixels, but this does not negative the fact that, as shown, as 
user makes a selection based upon criteria related to the dog and the archway. That is, regions 
are selected based on the positioning of the dog and the archway. As such, the positioning of the 
dog and the archway are considered criteria that are related to the 3-D object regions within the 
2-D image. 
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None of Appellant's arguments regarding the remaining claims raise other issues. 
Therefore, the rejection of these claims should be maintained for the reasons set forth above in 
the reprinted Final Rejection and the above rebuttal of Appellant's five arguments. 

(11) Related Proceeding(s) Appendix 

No decision rendered by a court or the Board is identified by the examiner in the Related 
Appeals and Interferences section of this examiner's answer. 

For the above reasons, it is believed that the rejections should be sustained. 
Respectfully submitted, 




